{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Testing extract_relevance_info_from_docs_with_conversation Function\n",
    "\n",
    "This notebook demonstrates how to use and test the `extract_relevance_info_from_docs_with_conversation` function from the ByzerLLM library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "质量系统在药品生产中扮演着至关重要的角色，它确保药品从研发到销售的每一个环节都符合严格的质量标准和法规要求。以下是质量系统的几个关键方面：\n",
      "\n",
      "1. **质量管理体系（QMS）**：\n",
      "   - 质量管理体系是一个组织内部用于确保产品和服务质量的框架。在药品行业，这通常包括一系列的标准操作程序（SOPs）、指导原则和流程，以确保药品的安全性、有效性和质量。\n",
      "\n",
      "2. **质量控制（QC）**：\n",
      "   - 质量控制涉及对原材料、中间产品和最终产品的物理、化学、生物和微生物测试。这些测试确保产品符合既定的质量标准。\n",
      "\n",
      "3. **质量保证（QA）**：\n",
      "   - 质量保证是一个更广泛的概念，它包括所有确保产品质量的系统活动。这不仅包括质量控制，还包括验证、变更控制、偏差管理、投诉处理和审计等。\n",
      "\n",
      "4. **风险管理**：\n",
      "   - 风险管理是识别、评估和控制潜在风险的过程，以确保这些风险不会对产品质量或患者安全造成负面影响。这包括使用工具如FMEA（失效模式与影响分析）和HACCP（危害分析和关键控制点）。\n",
      "\n",
      "5. **文件记录**：\n",
      "   - 详细的文件记录是质量系统的核心。这包括所有SOPs、批记录、验证报告、审计报告和变更控制记录等。这些文件不仅用于证明合规性，还用于追溯和持续改进。\n",
      "\n",
      "6. **培训**：\n",
      "   - 员工培训是确保质量系统有效运行的关键。所有员工都需要接受与其职责相关的培训，以确保他们了解并能够执行必要的质量控制和保证活动。\n",
      "\n",
      "7. **设施和设备管理**：\n",
      "   - 设施和设备需要定期维护和校准，以确保它们在良好的工作状态下运行。这包括洁净室管理、设备验证和环境监测。\n",
      "\n",
      "8. **供应商和物料管理**：\n",
      "   - 选择合格的供应商并管理物料的质量是确保最终产品质量的关键。这包括供应商的评估、审计和物料的接收、检验和存储。\n",
      "\n",
      "9. **产品放行和召回**：\n",
      "   - 产品放行是一个严格的过程，确保只有符合所有质量标准的产品才能投放市场。同时，建立有效的召回程序以应对可能的质量问题也是质量系统的一部分。\n",
      "\n",
      "10. **持续改进**：\n",
      "    - 质量系统应该是一个动态的、不断改进的过程。通过内部审计、管理评审和持续的质量改进项目，组织可以不断提高其产品和服务的质量。\n",
      "\n",
      "质量系统的有效实施不仅有助于确保药品的安全性和有效性，还能提高患者满意度，增强企业的市场竞争力。在药品行业，质量系统的合规性也是监管机构审查的重点。\n"
     ]
    }
   ],
   "source": [
    "from openai import OpenAI\n",
    "import json\n",
    "\n",
    "# 如果使用 HTTPS，请将 URL 改为 https://\n",
    "client = OpenAI(api_key=\"your_api_key_here\", base_url=\"http://127.0.0.1:8000/v1\")\n",
    "\n",
    "# messages = json.loads(open(\"/tmp/rag.json\").read())\n",
    "\n",
    "response = client.chat.completions.create(\n",
    "    messages=\n",
    "        [\n",
    "  {\n",
    "    \"role\": \"system\",\n",
    "    \"content\": \"这是历史聊天总结作为前情提要：质量系统在药品生产中至关重要，涉及质量管理、控制、风险管理、文件记录、培训、设>施设备、供应商物料管理及产品放行召回等多个方面。它确保药品从研发到销售的全过程符合质量标准和法规要求，保护消费者并提升企业信>誉。\"\n",
    "  },\n",
    "  {\n",
    "    \"role\": \"user\",\n",
    "    \"content\": \"给我详细介绍下质量系统\"\n",
    "    }],\n",
    "    model=\"deepseek_chat\",\n",
    ")\n",
    "print(response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "根据文档中的信息，董事会审议通过的利润分配预案为：以2,968,907,902股为基数，向全体股东每10股派发现金红利10元（含税），送红股0股（含税），以资本公积金向全体股东每10股转增7股。\n",
      "\n",
      "因此，这个数字是准确的。\n"
     ]
    }
   ],
   "source": [
    "from openai import OpenAI\n",
    "import json\n",
    "\n",
    "# 如果使用 HTTPS，请将 URL 改为 https://\n",
    "client = OpenAI(api_key=\"your_api_key_here\", base_url=\"http://127.0.0.1:8000/v1\")\n",
    "\n",
    "# messages = json.loads(open(\"/tmp/rag.json\").read())\n",
    "\n",
    "response = client.chat.completions.create(\n",
    "    messages=\n",
    "        [ \n",
    "  {\n",
    "    \"role\": \"user\",\n",
    "    \"content\": \"董事会审议通过的利润分配预案为：以2,968,907,902股为基数 这个数字准确么？\"\n",
    "    }],\n",
    "    model=\"deepseek_chat\",\n",
    ")\n",
    "print(response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare Test Data\n",
    "\n",
    "Let's create some sample documents and a conversation history to test our function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sample documents\n",
    "documents = [\n",
    "    \"The capital of France is Paris. It is known for its beautiful architecture and cuisine.\",\n",
    "    \"Python is a popular programming language used in data science and web development.\",\n",
    "    \"The Great Wall of China is one of the most famous landmarks in the world.\"\n",
    "]\n",
    "\n",
    "# Sample conversation history\n",
    "conversations = [\n",
    "    {\"role\": \"user\", \"content\": \"Tell me about France.\"},\n",
    "    {\"role\": \"assistant\", \"content\": \"France is a country in Western Europe known for its rich history and culture.\"},\n",
    "    {\"role\": \"user\", \"content\": \"What's the capital of France?\"}\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define and Test the Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The capital of France is Paris.\n"
     ]
    }
   ],
   "source": [
    "@byzerllm.prompt()\n",
    "def extract_relevance_info_from_docs_with_conversation(conversations, documents) -> str:\n",
    "    \"\"\"\n",
    "    使用以下文档和对话历史来提取相关信息。\n",
    "\n",
    "    文档：\n",
    "    {% for doc in documents %}\n",
    "    {{ doc }}\n",
    "    {% endfor %}\n",
    "\n",
    "    对话历史：\n",
    "    {% for msg in conversations %}\n",
    "    <{{ msg.role }}>: {{ msg.content }}\n",
    "    {% endfor %}\n",
    "\n",
    "    请根据提供的文档内容、用户对话历史以及最后一个问题，提取并总结文档中与问题相关的重要信息。\n",
    "    如果文档中没有相关信息，请回复\"该文档中没有与问题相关的信息\"。\n",
    "    提取的信息尽量保持和原文中的一样，并且只输出这些信息。\n",
    "    \"\"\"\n",
    "\n",
    "# Test the function\n",
    "result = extract_relevance_info_from_docs_with_conversation.with_llm(llm).run(conversations=conversations, documents=documents)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analyze the Results\n",
    "\n",
    "Let's analyze the output of our function to see if it correctly extracted the relevant information from the documents based on the conversation history."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Extracted relevant information:\")\n",
    "print(result)\n",
    "\n",
    "print(\"\\nDoes the extracted information answer the user's last question?\")\n",
    "print(\"User's last question:\", conversations[-1]['content'])\n",
    "print(\"Relevant information found:\", \"Yes\" if \"Paris\" in result else \"No\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test with Different Scenarios\n",
    "\n",
    "Let's test our function with different scenarios to ensure it works correctly in various situations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scenario 1: Question about a topic not in the documents\n",
    "conversations_scenario1 = [\n",
    "    {\"role\": \"user\", \"content\": \"What's the capital of Germany?\"},\n",
    "]\n",
    "\n",
    "result_scenario1 = extract_relevance_info_from_docs_with_conversation.with_llm(llm).run(conversations=conversations_scenario1, documents=documents)\n",
    "print(\"Scenario 1 Result:\")\n",
    "print(result_scenario1)\n",
    "\n",
    "# Scenario 2: Question about a topic in the documents, but not related to France\n",
    "conversations_scenario2 = [\n",
    "    {\"role\": \"user\", \"content\": \"Tell me about Python programming.\"},\n",
    "]\n",
    "\n",
    "result_scenario2 = extract_relevance_info_from_docs_with_conversation.with_llm(llm).run(conversations=conversations_scenario2, documents=documents)\n",
    "print(\"\\nScenario 2 Result:\")\n",
    "print(result_scenario2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "In this notebook, we've tested the `extract_relevance_info_from_docs_with_conversation` function with various scenarios. The function should effectively extract relevant information from the provided documents based on the conversation history and the user's last question. \n",
    "\n",
    "By analyzing the results, we can see how well the function performs in different situations, such as when the relevant information is present in the documents or when it's not available."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
